Program Synthesis in the Industrial World: Inductive, Incremental, Interactive

Programming by example (PBE), or inductive synthesis is a subfield of program synthesis where specification on a target program is provided in a form of input-output examples or, more generally, constraints on the output [9]. While research in PBE dates back as far as 1980s [8], its mass-market applications only appeared in the last decade. PBE has gained prominence due to its applicability in end-user programming – enabling end users to write small scripts for automating repetitive tasks from examples. The first PBE-based product was FlashFill – a system for automatic synthesis of string transformation scripts in spreadsheets [4]. It was incorporated in Microsoft Excel 2013, which made it available to hundreds of millions of end users. FlashFill was followed by FlashExtract, a system for automated data extraction from text files [6], which was deployed in Microsoft PowerShell and Azure Operational Management Suite. The team has since developed similar technologies for numerous data wrangling domains. Deployment of a mass-market industrial application of program synthesis is a challenging process. First, an efficient implementation requires non-trivial engineering insights, which are often overlooked during development of a research prototype. Second, the development process should be fast and agile, tailoring to versatile requirements of the product/user base. Third, the underlying synthesis algorithm should be understandable to engineers responsible for product maintenance. Initial implementations of FlashFill and FlashExtract violated all these conditions, requiring more than a year of research & development done by PhDs or graduate students. This prompted us to generalize the ideas of FlashFill, FlashExtract, and their successors into FlashMeta – a framework for automatic generation of domainspecific inductive synthesizers [9]. FlashMeta made development of domain-specific PBE applications scalable, allowing us to generate a production-ready synthesis library in a few weeks instead of months. In 2015, S. Gulwani’s PROSE (“PROgram Synthesis using Examples”) group at Microsoft [1] moved from Microsoft Research to Microsoft Data Group, establishing the first R&D group in PBE-based data wrangling at the company [3]. At Data Group, we are putting our algorithmic framework at the heart of new synthesis APIs, which enable various product teams to develop novel PBE features in their applications. Such an environment turns out fruitful for new problems and insights in program synthesis, which arise only in an end-user facing and industrially deployed applications. In this presentation, we want to share some of these problems, lessons, and potential solutions with the program synthesis community.